Recently, a collision prevention apparatus, a following distance control apparatus, etc., are installed in the car because of the increased awareness of safety of the car and improved computer performance. The apparatuses output a warning to the driver if the following distance falls within a certain range, and performs control of a brake device, a steering device, etc., so as to perform operations for automatically preventing the collision if the following distance becomes further shorter.
In order to perform such control, it is necessary to measure the following distance and a stereo camera goes into actual use as a distance measurement system. The stereo camera can generate a distance image in front of the vehicle, recognize an obstacle from the distance image, and then measure a distance to the obstacle.
The distance measurement using the stereo camera is described hereinafter in detail. FIG. 1 is a diagram for explaining a principle of the distance measurement by the stereo camera in which two cameras are disposed in parallel. A camera C0 with a focal length f, an optical center O0 and an imaging plane s0 is disposed such that a direction of an optical axis extends in a upper direction in FIG. 1, and a camera C1 with the same focal length f is deposed in parallel with and spaced by a distance B with respect to the camera C0 on the right side.
An image of a subject A which is located at a distance d from the optical center O0 of the camera C0 in the direction of the optical axis forms an image at P0 which is an intersection of a straight line AO0 and the imaging plane s0. With respect to the camera C1, the subject A forms an image at a point P1 on the imaging plane s1. Here, an intersection of a line and the imaging plane s1, which line passes through the optical center O1 of the camera C1 and is parallel with the line AO0, is indicated by P′0, and the distance between the point P′0 and the point P1 is indicated by p. The point P′0 is the same as the point P0 on the camera C0. The distance p indicates a deviation amount of the points on the images captured by these two cameras and is called parallax. Then, a triangle AO0O1 and a triangle O1P′0P1 are geometrically similar and thus the following formula 1 is given.d=Bf/p  (Formula 1)where B represents a base line length of a side of the triangle. If the base line length and the focal length f are known, the distance d can be determined based on the parallax p.
That is the principle of the distance measurement by the stereo camera in which two cameras are disposed in parallel; however, in order for this to hold, two cameras C0 and C1 must be disposed precisely, as shown in FIG. 1. Especially, if there is a rotation of the camera around a Y axis shown in FIG. 1 or a misregistration of the imaging sensor in a lateral direction with respect to the optical center, the image position in an X axis direction which is a lateral direction perpendicular to the Y axis in FIG. 1 changes. In this case, the misregistration is directly added to the parallax, which decreases accuracy of the measured distance.
In order to keep the necessary accuracy of the measured distance, it goes without saying that a precise adjustment is necessary at the time of manufacturing; however, it is also necessary to perform calibration because the arrangement can change slightly due to a change in temperature, a vibration, etc., if it is used for a long time. However, if it is necessary to perform calibration using a test chart or the like which is disposed at the known distance after the purchase, convenience of a user decreases greatly. In connection with this, several ways of performing the calibration without necessitating using the test chart or the like are proposed which utilize an image obtained by capturing an ambient natural scene (see JP3436074 B2, JP2001-169310 A, and JP2009-182879 A).
JP3436074 B2 discloses a technique in which a deviation of the camera imaging direction is detected as an arbitrary location such as a road, and the calibration is performed based on the detection results. Specifically, the same stationary feature object is recognized from the captured images at several locations for calibration on a traveling road, the deviation of the camera imaging direction between plural imaging means is determined based on the locations of the stationary feature object in the captured images at the several locations for calibration and the distances between the several locations for calibration, and the calibration is performed by performing correction based on the determined deviation of the camera imaging direction.
According to JP2001-169310 A, detection of the parallax of the stationary object is performed at two times; a travel distance of a vehicle between these two times is calculated; an error due to the deviation of parallelism of the optical axes between two cameras is determined as a parallax offset amount; and then the detected parallax is corrected with the parallax offset amount.
According to JP2009-182879 A, in order to enable calculating a calibration value of a camera parameter without performing an object recognition process, plural points of interest are set in input images captured by imaging means at a predetermined frame rate; parallaxes of the respective points of interest and a two-dimensional optical flow are calculated; a road surface parameter is calculated, which parameter indicates a relationship between a road surface included in the input image and the imaging means based on the parallaxes and vertical components of the points of interest; and then the calibration value of a camera parameter is calculated based on the optical flow, the parallaxes and the road surface parameter.
However, according to the method disclosed in JP3436074 B2, it is necessary to store a shape of the stationary subject. Thus, if there is no stationary subject stored, it is not possible to perform calibration. A green traffic signal or a traffic sign is used as the stationary subject; however, a shape or size of the traffic signal or a traffic sign is different depending on a region or a country and may be changed in the future. Therefore, according to this way of identification of the stationary subject with a registered shape, there is a possibility that the calibration cannot be performed successfully.
Further, similarly, according to the method disclosed in JP2001-169310 A, the shapes of the stationary subjects are registered and the calibration is performed by identifying the stationary subject based on the registered shapes. Thus, with respect to the stationary subject with a different shape due to the difference in a region or a country or a change in the future, it may be determined to be different from the registered stationary subjects and thus the calibration cannot be performed.
Further, according to the method disclosed in JP2009-182879 A, feature points are extracted from a light and shade pattern on the road surface; a distance measurement is performed using the feature points; road parameters indicating the position and the direction of the road surface are calculated; and the parallax offset is calculated based on the change in the road parameters. Thus, as long as the car travels, there is a high probability that a flat road surface exists immediately in front of the car, and thus the problem that the subject cannot be found is unlikely to occur. Further, since it is not necessary to recognize the subject with a particular shape, the problem that the calibration cannot be performed does not occur.
However, since a source of information for calculating the parallax offset is limited to the road surface, it is not possible to make advantageous use of stationary subjects other than a flat subject, such as a wall, a building, a pole, etc. Therefore, it cannot be applied to an application other than a on-vehicle application, such as an indoor application, in particular.
Further, since the light and shade on the road surface are fine and the number of the feature points decreases at a long distance, only a distance measurement at a short distance can be done and it cannot be utilized for the calibration. Depending on the purpose such as a following distance auto adjustment, etc., there may be a case where a preceding vehicle at a longer distance is desired to be measured. Thus, this may affect accuracy of the calibration.
For this reason, a calibration apparatus and a method are desired which can correct time-varying deviation (misregistration) of the stereo camera. In particular, such a calibration apparatus and a method are desired which can perform the calibration by utilizing various images captured during the traveling to detect a parallax offset with stability without depending on sensors other than a camera for measuring the own position information, such as a vehicle speed sensor, etc., and without relying on the existence of the subject with a particular shape such as a traffic signal or a flat road surface.